fix(runtime): narrow the block table, read Codex tool output, re-derive the shape counts - #229

The correctness and comment-accuracy half of DRC-4269. Every figure below was re-measured on the
local store rather than carried over; where my number disagrees with the finding's, I shipped mine
and say how it was measured. Two items are refuted with evidence and were not implemented.

1.1 _derive_block fires on quoted prose — FIXED

Re-measured across the whole local Claude corpus (3,774 transcripts, 2,828 with an assistant
message): the shipped table published 7 blocks. Structurally:

So 4 of 7 were the three phrases the finding names, all in a quoted or fenced span, two rendering
with no block language in the visible text.

waiting for your could not be fixed on its own: waiting for you is a prefix of it and is checked
first, so the listed waiting for your entry was dead code and giving it the first-person
treatment would have changed nothing. The fix is therefore two parts:

• the three bare phrases are replaced by self-state forms (i'm not permitted, i am not permitted, i don't have permission, i do not have permission, i'm waiting for your, i am waiting for your), which match 0 records today;
• an indicator must now end on a word boundary (_BLOCK_TRAILING_RE), which is what makes bare
  waiting for you stop matching the possessive, and _indicator_hit scans past a rejected hit so
  an earlier possessive cannot hide a later genuine hand-off.

Corpus effect: 7 blocks to 3, and the 3 are the clean ones above.

test_ordinary_reporting_prose_is_not_a_block gains four lines covering the three phrases plus the
prefix case. Two new tests pin the other direction:
test_the_first_person_replacements_still_report_a_real_block and
test_a_rejected_indicator_hit_does_not_hide_a_later_real_one.

1.2 The observer's stage half is unreachable on Codex — FIXED, with a measured shape

tool_result_text read record["message"], which a Codex rollout does not have. Measured before
the change: 0 of 458 local rollouts yielded a boot envelope. After: 10.

The payload shape was verified against real rollouts rather than guessed. Both spellings and both
value shapes are measured:

23 rollouts name a definition_dir anywhere; 18 carry a boot envelope on one of those two payload
types; 10 have it inside the spacedock_boot_scan_bytes head window.

The provenance rule is preserved: only a tool OUTPUT counts. A function_call's arguments field
carries the same JSON on the way in and is refused, which is the same distinction Pi's toolResult
role gate makes. Three new tests cover the four shapes, the negative twin, and the untrusted-value
guards. docs/design-spacedock.md gains decision S-6 with the numbers and the rejected alternative
(event_msg/item_completed, which turns out to be CommandExecution items whose definition_dir
mention is the command line rather than its output, carrying 0 envelopes).

1.3 The Codex dedup key degrades to text — FIXED

Measured over the whole rollout store, inside the observer's own head and tail windows:
599 of 784 Codex user-message records (76.4%) carry no payload.id. (The finding said 76.6%;
mine is 76.4% and the instrument is in the report notes.)

The fallback is now positional, and that needed the windows to stop overlapping first: they overlap
completely on any file the 400 KB tail read swallows whole, so a purely positional key would have
double-counted every record. _window_lines cuts them apart on byte offsets, after which position is
a genuine identity and the id key is left doing the one job it is good at, collapsing a resumed
transcript's replayed block.

tests/_codex_message takes payload_id=None and writes the record with no id key at all.
test_a_repeated_codex_prompt_with_no_payload_id_keeps_the_newest is the regression;
test_a_replayed_record_is_still_deduped_by_its_own_id guards the other side.

One sub-claim refuted: there is no 72.6% figure anywhere in the tree (grep -rn 72.6 finds only
test digests). The _dedup_key docstring's inherited figure is "0 of 8,312 Claude and 0 of 14,389
Codex records carry a top-level id", which is a different claim and still true. The docstring now
carries the measured 76.4% with its population named.

1.4 instruction_from's unconditional title_is_prompt suppression — REFUTED, not implemented

The premise is that replacing the Python suppression with the frontend echo test would surface the
withheld characters. It would not. Measured by running instruction_from both ways over all 458
rollouts and evaluating nextInstructionEchoes (from web/next/next-boot.js) in Python:

today     {'none': 230, 'agent': 43, 'earlier': 8}
proposed  {'asked': 213, 'none': 17, 'agent': 43, 'earlier': 8}
asked lines: suppressed by the frontend 213, rendered 0, of which title was clipped 151

All 213 new lines are suppressed, including the 151 clipped-title rows that are exactly the
population the finding wanted surfaced. nextInstructionEchoes's second clause
(head.endsWith("…") && line.startsWith(head.slice(0, -1))) is written for that case and is
deliberate: line 1 clips at 80 and line 2 at 140, so one prompt reaches them as two strings.

Implementing the Python half alone would emit 213 payload fields that render nowhere and put the
same policy in two places, which is the opposite of the item's stated point. I did not implement it.
The instruction_from docstring now records the measurement so the finding is not re-derived.

web/ follow-up to file (a sibling owns that file, so it is not in this PR): relax
nextInstructionEchoes so a strictly longer extension of a clipped title is kept rather than
suppressed, then delete title_is_prompt. That is the change that surfaces the withheld characters,
and it belongs in the PR that owns web/.

3.1 The window test is vacuous and the route test is Pi-shaped — FIXED

• test_the_head_window_and_the_tail_window_are_both_read now asserts the block as well as the
  goal. The goal lives only in the head and the block only in the tail, so it fails when either
  window is not read. test_the_tail_assertion_is_load_bearing commits the falsification: with
  read_tail stubbed to [] the goal survives and the block disappears, so if a later edit makes
  the block reachable from the head, that test goes red rather than the first one going quiet.
• The /api/observe route test wrote a Pi-shaped type: "message" record into the Claude store,
  a shape that occurs 0 times in 3,774 real Claude transcripts. It now writes Claude-shaped records
  and asserts the block too.
• test_the_route_answers_for_a_codex_rollout is new: a realistic rollout with session_meta,
  id-less response_item messages, the resolver matching on session_meta rather than a filename
  stem, and the sidecar written.

Non-vacuity was proved by breaking the code and watching the intended test go red, one edit at a
time (7 for 7). The exact edits are in the report notes.

3.3 No committed derivation script — FIXED

scripts/derive_prompt_shapes.py re-derives every count in the harness-injected-prompt comment
block from a store the caller points it at. Every figure now in records.py is one of its outputs.

It prints counts and shape names only, never prompt text. That constraint is in the module
docstring and enforced by _safe_label, a whitelist rather than a filter: a label must be a short
markup tag name, a literal already present in records.py's own vocabularies, or one of a fixed set
of column headings, and anything else raises UnsafeLabelError. A tag name longer than the bound is
aggregated under a structural label rather than printed, and rather than raised on, so a weird
transcript cannot make the script unusable. Discovering a new prose prefix is deliberately out of
scope, because it cannot be done without printing prose.

scripts/tests/test_derive_prompt_shapes.py seeds a distinctive prompt into a fixture store and
asserts the whole stdout carries none of it, asserts the guard rejects prompt-shaped labels, and
asserts the script's _turn_signal reimplementation agrees with the real one on seven record
shapes. Wired into the pre-PR list in AGENTS.md and into both coverage blocks in
quality-gate.yml; the script itself reports 89.3% coverage.

4.1 Three wrong figures in transcripts.py — FIXED, with my own measurements

The finding said 83.1 ms and 78 of 88; I measure 84 ms and 79 of 89. The docstring now separates the
two AgentMessage numbers, because conflating "the shape exists" with "the code reads it" was the
defect. The UserMessage branch is kept and now says plainly that it changes no reading today.

4.8 Comment-block provenance — FIXED

The block header now names the exact record population behind each half:

• Codex user set: the union of 1,007 event_msg/user_message texts and 1,734 response_item
  message texts with role: "user", across 458 rollouts. That union double-counts: 974 of the
  1,007 event_msg prompts have a matching response_item. So the counts are occurrences of a
  shape, not prompts, and the comment says so. (Equality matching scores 850; the 974 uses a
  400-character head match, because extract_text caps a block-list join at 2,000 characters while
  an event_msg's message is one uncapped string.)
• Codex developer set: 1,826 response_item message texts with role: "developer".
• Claude: 211,669 user-role texts across 3,774 transcripts on the collector's own glob.

One sub-claim shipped differently. The finding says "six Claude entries totalling 5,367
occurrences count records _turn_signal already refuses". Running the real records._turn_signal
over every leading-tag record gives four entries and 1,917 of the 5,393 occurrences:
local-command-caveat 1,065 of 1,065, local-command-stdout 621 of 621, system-reminder 223 of
227, channel 8 of 8. task-notification, teammate-message, bash-input, bash-stdout and
local-command-stderr are refused 0 times. I shipped my numbers.

4.9 Developer-tag counts are containment counts — FIXED

Confirmed exactly. Leading versus containment on the same corpus:

The documented set was a mix of both, which is exactly how collaboration_mode looked small. The
counts are now leading counts, labelled as such. The two that lead 0 records are kept for the
documented asymmetry (an unlisted tag renders markup as a person's words; a listed one that never
arrives costs nothing) but are labelled 0 leading (N contained) — defensive only rather than left
looking measured. turn_aborted leads 52 developer-role records and is noted on its user-set entry.

4.10 The [Image arithmetic — FIXED

Three populations, not one plus an anecdote:

The old comment said "385 of 386" and called [Image source: "the 386th". It is a separate
population of 9, and the reason the separator is a character class is [Image #N], where a space
follows Image. (The finding said 386, 9, 134/133; my corpus is one record larger on two of the
three.) The Codex <image> figure reproduces exactly at 36, all carrying operator text.

4.11 Warmup — 97 over a stated denominator, and subagent-only

I measure 97, not 85, over the Claude population the block header names. The comment now states
that denominator instead of leaving it implied, and adds the facts that matter: all 97 are
isSidechain records, 0 arrive on a Codex rollout, and _turn_signal already refuses 12 of the 97.
That is likely where the finding's 85 comes from, but I did not want to ship a number I could not
reproduce, so the comment carries mine with its population named.

4.12 Pointer comments for two duplications — FIXED

• _turn_signal's hardcoded ("<local-command-stdout>", "<local-command-caveat>") now carries a
  note that both names are also in _CLAUDE_USER_TAGS, that this is a record rule and that is a
  text rule, that the tag set covers seven further names this must not refuse, and that a name added
  here belongs in both.
• observer._is_generic_opener and records.injected_prompt point at each other, with the reason
  they are separate (a goal rule versus a record rule) written once in the observer.

Nits

Fixed:

• Leading U+FEFF / U+200B / U+2060 (the one with a security shape). str.strip() removes
  whitespace and none of these is whitespace to Python, so one character in front of a record
  defeated the leading-tag regex, the prose prefixes and the whole-body set at once, and
  injected_prompt answered "the operator said this" about harness machinery. strip_prompt_wrappers
  now trims _UNSAFE_CHARS minus C0/DEL, plus U+2060 and U+FEFF; U+200C and U+200D stay out because
  they are orthographic. Test covers six lead shapes across three branches plus the other direction.
• Double truncation deleting the ellipsis. _derive_goal_deterministic scrubbed at the cap, not
  cap + 1, so a hard-cut goal lost its …. Measured: 8 of the 1,295 goals the local Claude
  corpus publishes, of which 269 clip at all. (The nit said 11 of 1,469.) Same + 1 and same reason
  as records.instruction_line.
• shorten_paths running undocumented. One clause added where the goal is rendered. No number,
  because I did not measure the 69 and would not vouch for it.
• _SHARED_MESSAGE_HARNESS. The comment now says the dict miss is the mechanism rather than an
  accident, and that the Droid half of the name is aspirational because resolve_transcript answers
  for claude, codex and pi only.

Skipped, with the reason:

• Case asymmetry between the tag and prose branches. Making the prose branch case-insensitive
  would newly reject an operator writing "warmup the cache", which is the false positive
  _INJECTED_PROMPTS is matched-whole to avoid. 0 corpus incidence either way.
• The early return that skips the prose check after an unlisted leading tag. Provably inert: a
  body that matches _PROMPT_LEADING_TAG_RE starts with <, and no listed prose prefix or
  whole-body literal starts with <. The branch cannot change an answer.
• [Image processing] peeled as a marker. Narrowing the wrapper regex to the three measured
  spellings risks dropping an unmeasured marker variant, and the failure mode of that (publishing a
  marker as a goal) is the one this code exists to prevent. 0 corpus incidence.
• The quadratic strip loop. As the finding says: no caller, extract_text caps at 2,000
  characters, 42 ms at 8,000 stacked markers. Not worth the complexity.
• The unpinned <command-args> branch. I could not identify a specific defect from the
  description without guessing which regex and which branch were meant, and there is already a
  fixture exercising an empty <command-args>. Not acted on rather than acted on speculatively.
• NO_GOAL_REASON naming a never-occurring condition. A provenance note only, and I would have
  had to run another corpus pass to vouch for a number. The docstring already explains the sentinel
  bounds the model.
• No version signal in /api/health or write_state. Scope creep, as the finding allows.

Verification

Full pre-PR suite run once on this tree. ruff check, ruff format --check, mypy --strict,
lint_embedded.py --allow-missing-node, validate_plugins.py, bump_version.py --current all
clean. Dashboard suite 1,919 tests OK (1 skipped, pre-existing). Scripts modules 190 tests OK.
Coverage total 90.7%, fail_under 73. No manifest file is touched, so version-guard has nothing
to see.